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Friday, May 18, 2012 


Open Access Mandates: Ensuring 
Compliance 


Three years ago, | wondered aloud how we should judge the success of an Open 
Access (OA) mandate. In particular, | wondered what level of compliance it was 
reasonable to expect a mandate to attain, and pointed out that OA advocates 
believe it is possible to achieve a 70% compliance rate within two years, if the 
mandate is compulsory. 


It is therefore striking that, although the Wellcome Trust has had a compulsory 
mandate in place since 2006, The Times Higher Education reported in March that 
the Trust is still only achieving a 55% compliance rate. As a result, The Times 
Higher added, Wellcome’s head of digital services Robert Kiley wants to get tough 
on those who do not comply. The current situation, said Kiley, is “simply not 
acceptable”. 


Elsewhere, Nature reported that of the 55% of researchers who comply with 
Wellcome’s mandate, 85% do so by means of Gold OA. One benefit of Gold OA, 
presumably, is that the author can hope to pass responsibility for compliance over 
to the publisher. While this does require paying a Gold OA fee, the Trust will pay 
this fee for the researcher. However, that 85% figure might seem to suggest that 
authors are more reluctant to comply with a mandate than some assume. 


In my 2009 blog post | also reported that when, in 2008, the US National Institutes 
of Health (NIH) upgraded its request that researchers deposit their published 
Papers into PubMed Central (PMC) into a requirement, compliance accelerated to 
49% by the end of the year (from 19%). 
wt INS», In order to establish what level of compliance the NIH is 

O %, achieving today, | contacted its Office of Extramural 

Ee Research (ORE). Below is the (slightly edited) transcript 

$ ay of the email conversation | had, which was split over a 
(e@) ae number of emails. 

HENS 
The upshot is that the NIH is now achieving 75% 
compliance. 


This invites a question: Why is NIH proving more successful in achieving compliance 
than the Wellcome Trust? Here are some quick thoughts: 


¢ NIH is the largest public funder of medical research in the world, and so 
perhaps has greater authority than the Wellcome Trust 


¢ NIH has been more proactive in ensuring compliance 


e Where the Wellcome Trust insists that any embargo imposed by a 
publisher is no longer than six months, the NIH still permits embargoes 
to be up to 12 months (so perhaps publishers are more willing to post 
NIH-funded papers in PMC?) 


As | say, these are just initial thoughts. | welcome comments from others. 


One thing | find noteworthy is that, while the NIH clearly takes a close interest in 
compliance levels, it does not collect statistics on enforcement actions. With the 
growth in compliance now apparently levelling off at NIH, this would surely be 
useful information — and not just for NIH itself, but also for other funders like the 
Wellcome Trust and Research Councils UK (RCUK), both of whom are currently 
looking to beef up their open access policies. 


Q&A with NIH 


RP: Can you give me some information on the current compliance rates for the 
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NIH Public Access Policy, and explain how the NIH ensures compliance? 


NIH: OSTP has submitted a report to Congress that included a current overview of 
the NIH policy, compliance level and impacts on Page 12. 


Our current compliance efforts have been focused on outreach and engagement 
with institutions and publishers, as outlined on pages 3 and 4 of David Lipman's 
testimony to Congress in 2010. 


RP: Ok, so after reading these documents | conclude the following: 
1. The initial voluntary policy saw compliance rates of 19%. 


2. The mandatory policy (introduced in April 2008) saw compliance grow to 
49% by the end of 2008, 70% by the end of 2009 and currently stands at 75%. 


3. In order to increase compliance the NIH has a) engaged in awareness 
programs; b) improved its ability to track papers resulting from NIH research 
awards; c) developed new systems to assist sponsored research offices at 
universities and medical research centres in tracking their compliance; d) 
improved the submission process for authors by i) co-opting publishers to 
deposit for authors and ii) introducing the NIH Manuscript Submission System. 


NIH: To be clear, we introduced the NIH manuscript submission system in 2005. We 
have made modifications and improved our guidance, but it is not new. 


We don't think it is accurate to say that we have ‘co-opted’ publishers, as the NIH 
policy does not apply to them. 


RP: Why do you think publishers agree to deposit papers for authors? 
NIH: We generally hear three reasons why some publishers take an active role: 


1) They want to be on PMC because of its high visibility, integration with other 
resources, and secure and free archiving — many journals do this for all their 
content, regardless of whether it falls under the NIH public access policy 


2) They want to provide a service to their authors 


3) They want greater control over the version posted to PMC (some want only the 
published version posted, and others want only the final peer-reviewed manuscript 
posted) 


Green or Gold? 


RP: Can you say what percentage of the papers deposited in PubMed Central 
are papers that have been published in a subscription journal and then self- 
archived (i.e. Green OA) and what percentage have been published in an OA 

journal for a fee (i.e. Gold OA)? 


NIH: We do not characterize PubMed Central (PMC) deposits in terms of green and 
gold Open Access (OA). PMC is more than a repository for papers resulting from 
NIH-funded research. See here. 


All 2.4 million PMC articles are available without charge on PMC. This number 
includes about 225,000 author manuscripts, and the rest are final published 
versions that have been provided to PMC by publishers. 


We do not have a simple way of generating counts of which papers are from 
subscription based journals and which are not. However, to put the above numbers 
in context, note also that close to half of the articles in PMC are from the digitized 
(scanned) issues of journals from the period between the mid-1800s and the mid to 
late 1990s — predating the birth of OA journals and the NIH public access policy. 


About 450,000 articles make up the PMC OA subset. The term Gold OA generally 
refers to articles that are available under a very open license (such as CC-BY) and 
involve an open access fee charged to authors. The articles in the PMC OA subset 


come with a variety of licenses, not just CC-BY. 


In addition, some of these OA articles are from journals that, to our knowledge, 
don't charge an OA fee — the Rockefeller Press journals, for example here. 


Enforceable? 


RP: I came across a message on the Liblicence mailing list recently from 
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someone who appeared to have concluded that there is no real force behind 
the NIH policy. Would it be accurate to say that the NIH currently has no way 
of enforcing compliance? 


NIH: The NIH Public Access policy is a term and condition of award. As we state in 
our FAQ: A grantee's failure to comply with the terms and conditions of award may 

cause NIH to take one or more enforcement actions, depending on the severity and 
duration of the non-compliance. NIH will undertake any such action in accordance 

with applicable statutes, regulations, and policies. 


NIH generally will afford the grantee an opportunity to correct the deficiencies 
before taking enforcement action unless public health or welfare concerns require 
immediate action. 


However, even if a grantee is taking corrective action, NIH may take proactive 
action to protect the Federal government's interests, including placing special 
conditions on awards or precluding the grantee from obtaining future awards for a 
specified period, or may take action designed to prevent future non-compliance, 
such as closer monitoring. See Enforcement Actions in the NIH Grants Policy 
Statement (11/03). 


RP: Can you tell me how many researchers have received research funds for 
the last five years, and then give me annual figures on enforcement actions 
that have been taken against researchers who have failed to comply with the 
public access policy. E.g. how many enforcement letters have been sent out, 
how many researchers have had special conditions placed on their awards, 
and how many have been precluded from obtaining future awards for a 
specified period (and for how long a period)? 


NIH: We funded over 228,000 grants in the past 5 years, which support hundreds of 
thousands of investigators. NIH public access policies have been developed 
centrally, and compliance activities are carried out by individual NIH institutes and 
centers. 


There are currently terms and conditions which require institutions to comply with 
the public access policy. When an investigator is found out of compliance with the 

NIH public access policy, the NIH institute or center in which the award resides will 
send out letters to bring the investigator into compliance. 


NIH does not currently collect statistics on compliance efforts. 


Posted by Richard Poynder at 15:27 [ae] 


Labels: Mandates, NIH, Open Access 


7 comments: 


Bill Hooker said... 


Thanks for following up on this, Richard. (| think my request for actual 
data was bounced off the list, dunno why.) 


The link to the OSTP report (first A of Q&A) isn't working. 
May 18, 2012 11:59 pm 


Richard Poynder said... 
bse Thanks for pointing out the dead link Bill. It should be fixed now. 


Your message was posted on Liblicense, and indeed your guess seems 
to have been pretty accurate! 


May 19, 2012 6:30 am 


Richard Poynder said... 
hiss Cameron Neylon posted this comment on Googlet: 


Basically the reason for higher compliance with NIH as far as | can tell is 
that grant reporting is coupled into deposition. So you can't include a 
paper in your grant report unless it's in PMC. | haven't actually seen this 
in action myself but it's consistent with the fact that places like Liege get 
very high compliance with institutional mandates when reporting and 
review is technically tied into compliance. 


May 19, 2012 9:39 am 


Stevan Harnad said... 


Where is the Open Access 
Foundation? 

While the term Open Access 
(OA) has its origins in the 
2002 Budapest Open Access 
Initiative (BOAI), the concept 
and practice of OA has bee... 
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How to Maximize Compliance With Funder OA Mandates: 
Collaborate With Institutional Mandates 


Here are my hypotheses as to why neither NIH nor the Wellcome Trust 
(WT) has a compliance rate of 100% (and why NIH has a somewhat 
higher compliance rate): 


1. How To Comply. Both the NIH and WT mandates designate Gold OA 
publishing as one of the means of fulfilling the mandate, instead of 
uniformly designating fundee self-archiving as the sole means of 
compliance (whether or not the fundee publishes in a Gold OA journal. 


2. Who Complies. Funder mandates only apply to fundees: only 
fundees are bound by them. Yet fulfillment can be done by either 
fundees or non-fundees (publishers, especially in the case of WT), 

instead of uniformly designating fundee self-archiving as the sole means 
of compliance. 


3. When To Comply. The designated timing for compliance with both 
mandates is not immediately upon publication -- instead of uniformly 
designating fundee self-archiving immediately upon publication as the 
sole means of compliance (even if the self-archived draft is not made 
immediately OA). As noted, it is in publishers’ interests to make 
compliance as delayed as possible, and to leave it in their hands rather 
than the fundees'. 


4. What Version To Deposit. It contributes to the delay in compliance 
and the ambiguity as to who is fulfilling the mandate (the fundee or the 
publisher) if compliance can wait for the publisher's PDF instead of 
uniformly designating fundee self-archiving of the refereed final draft 
immediately upon publication as the sole means of compliance (even if 
the self-archived draft is not made immediately OA and the publisher's 
PDF is optionally deposited later). 


5. Where To Deposit. Both NIH and WT grants stress direct deposit in 
PubMed Central (PMC), instead of uniformly designating fundee self- 
archiving of the refereed final draft in the fundee's own institutional 
reposiitory immediately upon publication as the sole means of 
compliance (even if the self-archived draft is not made immediately OA 
and the publisher's PDF is optionally deposited later), thereby recruiting 
fundees' institutions to monitor and ensure compliance with the 
fulfillment conditions of the grant (as institutions are always very eager to 
do!). 


Institututional ID/OA Mandates Work. None of these delays, 
ambiguities or uncertainties applies to (effective) institutional mandates 
such as U. Liege's model ID/OA (immediate-deposit/optional-access) 
mandate. Not only can author self-archiving in the institutional repository 
be designated by institutions as the sole means of submitting research 
for institutional reporting and performance assessment (as Cameron 
Neylon correctly points out), but institutions are in a position to monitor 
deposits continuously, not just when a research project grant (which may 
last for years) has elapsed. 


Mutual Reinforcement Between Institutional and Funder Mandates. 
In addition, designating institutional repository self-archiving as the 
means of compliance for both funder and institutional mandates 
motivates institutions to adopt self-archiving mandates of their own, for 
all of their research output, in all disciplines, not just NIH- or WT-funded 
research. (Institutions are the universal providers of all published 
research, funded and unfunded.) Funder mandates designating 
institutional deposit make institutional and funder mandates convergent 
and mutually reinforcing -- rather than divergent and competitive, as 
funder mandates requiring direct institution-external deposit in PMC 
(instead of just automated harvesting or export from institutional 
repositories) do. 


Effective Institutional Mandates Can Generate 100% OA Globally. 
The Liege model institutional |D/OA mandate really works. If funders and 
institutions worldwide collaborate, 100% OA can be reached not just for 
NIH and WT funded research but for all research. 


May 19, 2012 12:00 pm 


Mike Taylor said... 


| have to say | find it incomprehensible that any mandate's compliance 
rate is not 100%. 
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If | pay a merchant to deliver a new oven to my home and install it, but 
they just dump it outside my front door and walk away, then the very 
least sanction | will impose is never giving them my money again -- that's 
if | don't seek legal redress. 


So if NIH or Wellcome pays someone to carry out a research programme 
and deposit the resulting papers in PubMed, and if they don't bother to 
do that, then the very least sanction the funder imposes should be to 
never give that group money again. 


| don't see how this could be controversial. If you're paid to do 
something, you do it. 


May 21, 2012 12:50 pm 


dave said... 


‘e) Thanks for the follow up. This sounds like what many expected-- "NIH 
does not currently collect statistics on compliance efforts." 


In my mind | was contemplating whether a FOIA request would do any 


good here, but it sounds like with the lack of record keeping would mean 
it would produce little additional information. 


May 24, 2012 5:26 pm 


Alicia Wise @wisealic said... 


a) Hi Richard, 


Apologies for coming late to this discussion. 

We estimate that more than 60% of manuscripts posted on PMC are 
published in journals where the publishers deposit manuscripts on behalf 
of authors. 

* What percentage of NIH compliance comes from publisher deposit? 

* What is the compliance level for authors depositing on their own? 
Thanks for your insight, 

Alicia 

Dr Alicia Wise 

Director of Universal Access 


Elsevier 
@wisealic 


June 01, 2012 10:45 pm 
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